Quality is in the eyes of the reviewer: a report on post-editing quality evaluation

As part of a larger research project exploring correlations between productivity, quality and experi-ence in the post-editing of machine-translated and translation-memory outputs in a team of 24 pro-fessional translators, three reviewers were asked to review the translations/post-editions completed by these translators and to fill in the corresponding quality evaluation forms. The data obtained from the three reviewers’ evaluation was analyzed in order to determine if there was agreement in terms of time as well as in number and type of errors marked to complete the task. The results show that there were statistically significant differences between reviewers, although there were also cor-relations on pairs of reviewers depending on the provenance of the text analyzed. Reviewers tended to agree on the general number of errors found in the No match category but their agreement in Fuzzy and MT match was either weak or there was no agreement, perhaps indicating that the origin of the text might have influenced their evaluation. The reviewers also tended to agree on best and worst performers, but there was great disparity in the translators’ classifications if they were ranked according to the number of errors

Productivity and quality in the post-editing of outputs from translation memories and machine translation

Machine-translated segments are increasingly included as fuzzy matches within the translation-memory systems in the localisation workflow. This study presents preliminary results on the correlation between these two types of segments in terms of productivity and final quality. In order to test these variables, we set up an experiment with a group of eight professional translators using an on-line post-editing tool and a statistical-based machine translation engine. The translators were asked to translate new, machine-translated and translation-memory segments from the 80-90 percent value range using a post-editing tool without actually knowing the origin of each segment, and to complete a questionnaire. The findings suggest that translators have higher productivity and quality when using machine-translated output than when processing fuzzy matches from translation memories. Furthermore, translators' technical experience seems to have an impact on productivity but not on quality

What do professional translators think about post-editing

As part of a larger research project on productivity and quality in the post-editing of machine-translated and translation-memory outputs, 24 translators and three reviewers were asked to complete an on-line questionnaire to gather information about their professional experience but also to obtain data on their opinions about post-editing and machine translation. The participants were also debriefed after finalising the assignment to triangulate the data with the quantitative results and the questionnaire. The results show that translators have mixed experiences and feelings towards machine-translated output and post-editing, not necessarily because they are misinformed or reluctant to accept its inclusion in the localisation process but due to their previous experience with various degrees of output quality and to the characteristics of this type of projects. The translators were quite satisfied in general with the work they do as translators, but not necessarily with the payment they receive for the work done, although this was highly dependent on different customers and type of task

Exploring Machine Translation on the Web

En aquest article s'explora la traducció automàtica a la web, la seva història i la investigació actual. També s'hi examinen breument quatre motors de traducció en línia gratuïts, en funció de les combinacions lingüístiques que ofereixen, la longitud dels textos que accepten i els formats dels documents admesos, així com la qualitat de la traducció automàtica que generen directament.En este artículo se explora brevemente la traducción automática en la web, su historia y la investigación actual. Se examinan cuatro motores de traducción en línea gratuitos en base a las combinaciones lingüísticas que ofrecen, la longitud de texto que aceptan y los formatos de documentos soportados, así como la calidad de la traducción automática que generan directamente.This article briefly explores machine translation on the web, its history and current research. It briefly examines as well four free on-line machine translation engines in terms of language combinations offered, text length accepted and document formats supported as well as the quality of their raw MT output

Productivity and quality in the post-editing of outputs from translation memories and machine translation

This study presents empirical research on no-match, machine-translated and translation-memory segments, analyzed in terms of translators’ productivity, final quality and prior professional experience. The findings suggest that translators have higher productivity and quality when using machine-translated output than when translating on their own, and that the productivity and quality gained with machine translation are not significantly different from the values obtained when processing fuzzy matches from a translation memory in the 85-94 percent range. The translators’ prior experience impacts on the quality they deliver but not on their productivity. These quantitative findings are triangulatedwith qualitative data from an online questionnaire and from one-to-one debriefings with the translators. Este estudio presenta una investigación empírica sobre la traducción de segmentos nuevos y aquellos procesados con traducción automática y memorias de traducción analizados en relación a la productividad, calidad final y experiencia profesional de un grupo de traductores. Los resultados sugieren que los traductores obtienen una productividad y calidad más altas cuando procesan segmentos de traducción automática que cuando traducen sin ninguna ayuda y que dicha productividad y calidad no son significativamente diferentes a la que se obtiene cuando procesan coincidencias parciales de una memoria de traducción (del 85 al 94 por ciento). La experiencia profesional previa de los traductores influye en la calidad pero no así en la productividad obtenidas. Los resultados cuantitativos se triangulan, además, con datos cualitativos obtenidos a través de un cuestionario en línea y de entrevistas individuales realizadas a los tra

The role of professional experience in post-editing from a quality and productivity perspective

In this chapter, we present results on the impact of professional experience on the task of post-editing. These results are part of a larger research project where 24 translators and three reviewers were tested to obtain productivity, words per minute, and quality data, errors in final target texts, in the post-editing of machine translation (MT) and fuzzy match segments (in the 85 to 94 range). We will discuss here the results on the participants’ experience according to their responses in a post-assignment questionnaire and explain how they were grouped into different clusters in order to correlate firstly the experience with speed according to the words per minute in the different match categories: Fuzzy matches, MT matches (MT output) and No match and secondly, to correlate them with the quality provided by measuring the errors marked by the three reviewers in each match category. Finally, conclusions will be drawn in relation to the experience and the resulting speed and number of errors

Productivity and quality in MT post-editing

Machine-translated segments are increasingly included as fuzzy matches within the translation-memory systems in the localisation workflow. This study presents preliminary results on the correlation between these two types of segments in terms of productivity and final quality. In order to test these variables, we set up an experiment with a group of eight professional translators using an on-line postediting tool and a statistical-base machine translation engine. The translators were asked to translate new, machine-translated and translation-memory segments from the 80-90 percent value using a post-editing tool without actually knowing the origin of each segment, and to complete a questionnaire. The findings suggest that translators have higher productivity and quality when using machinetranslated output than when processing fuzzy matches from translation memories. Furthermore, translators’ technical experience seems to have an impact on productivity but not on quality. Finally, we offer an overview of our current research

Pre-editing and post-editing

This chapter provides an accessible introductory view of pre-editing and post-editing as the starting-point for research or work in the language industry. It describes source text pre-editing and machine translation post-editing from an industrial as well as academic point of view. In the last ten to fifteen years, there has been a considerable growth in the number of studies and publications dealing with pre-editing, and especially post-editing, that have helped researchers and the industry to understand the impact machine translation technology has on translators’ output and their working environment. This interest is likely to continue in view of the recent developments in neural machine translation and artificial intelligence. Although the latest technology has taken a considerable leap forward, the existing body of work should not be disregarded as it has defined clear research lines and methods, as it is more necessary than ever to look at data in their appropriate context and avoid generalizing in the vast and diverse territory of human and machine translation

Machine translation and post-editing training as part of a master’s programme

This article presents a description of a machine translation (MT) and post-editing course (PE), along with an MT project management module, that have been introduced in the Localisation Master’s programme at Universitat Autònoma de Barcelona in 2009 and in 2017 respectively. It covers the objectives and structure of the modules, as well as the theoretical and practical components. Additionally, it describes the project-based learning approach implemented in one of the modules, which seeks to foster creative and independent thinking, teamwork, and problem solving in unfamiliar situations, with a view to acquiring transferable skills that are likely to be in demand, regardless of the technological advances taking place in the translation industry

Creative skills development: training translators to write in the era of AI

Developers of generative artificial intelligence systems promote the idea of personal assistants for various tasks, including translation and authoring creative content. As a consequence of these developments, the topic of “human” creativity has moved centre stage. Acknowledging similarities between translation and creative writing, this article offers a critical discussion of intersecting areas and suggests a framework for creative skills couched in the tradition of social sciences research. As a practical application with pedagogical impact, the paper presents a new module on writing specifically designed for translators. As is argued, the conceptual design, content, mode of delivery and evaluation of potential pedagogical benefits may be replicable in other pedagogical settings at undergraduate or postgraduate level. The role of technology is also problematised, indicating how writing may be augmented by using tools. Ideally, this is to be done in a context where creativity upskilling can equip students with the ability to (de)select context-appropriate solutions, that is, to use convergent and divergent thinking, ultimately preparing them to play a fundamental role in a rapidly evolving digital world